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Preface 


As  an  Air  Force  officer  involved  in  managing  the 
acquisition  of  modern  weapon  systems,  I  have  always  felt 
that  my  electrical  engineering  undergraduate  degree  was  of 
definite  benefit.  My  specialty  in  electromagnetic  fields 
and  waves  helped  me  to  feel  comfortable  in  dealing  with 
technical  issues  associated  with  radar  and  communications 
systems.  At  the  same  time,  however,  I  have  realized  that 
much  of  the  technical  knowledge  I  have  acquired  was  not 
gained  from  my  academic  education.  I  attribute  my  own 
technical  expertise  more  to  an  intrinsic  ability  to  assimi¬ 
late  information  than  to  my  academic  background. 

Being  aware  of  the  significant  differences  among 
various  specialties  within  an  electrical  engineering  degree 
program  and  the  broader  disparities  between  different 
types  of  engineering  curricula,  I  am  disturbed  by  those  who 
classify  individuals  into  just  two  groups — engineers  and 
those  who  are  not.  I  have  known  Air  Force  acquisition 
personnel  with  engineering  degrees  who,  in  my  judgement, 
did  not  do  well  at  absorbing  technical  information  and 
applying  it  to  their  job.  On  the  other  hand,  I  have  met 
a  number  of  individuals  without  technical  degrees  who  were 
able  to  deal  very  effectively  with  complex  technical  issues. 


My  personal  view  is  that  those  directly  involved 


in  managing  the  acquisition  of  modern  military  systems 
need  to  thoroughly  understand  the  technical  aspects  of  the 
system  being  acquired  in  order  to  make  the  proper  manage¬ 
ment  decisions.  A  program  office  comprised  of  "non¬ 
technical"  business  managers  assisted  by  an  engineering 
support  staff  is  suboptimal,  in  my  opinion. 

Holding  this  view,  I  was  surprised  to  find  that  a 
proposed  new  regulation  addressing  career  development  and 
selection  of  Air  Force  acquisition  managers  barely  men¬ 
tioned  technical  considerations.  I  was  further  concerned 
upon  discovering  that,  according  to  other  officer  personnel 
regulations,  a  prerequisite  for  becoming  a  Program  Manager 
is  based  upon  one's  undergraduate  education. 

This  study  focused  on  technical  competence  of  the 
Air  Force  System  Program  Office  (SPO)  director;  the  program 
management  job  I  consider  to  be  at  the  pinnacle  of  the 
acquisition  management  profession.  It  attempted  to  dis¬ 
cover  how  technical  competence  in  program  managers  is 
related  to  academic  education  and  to  intrinsic  aptitude. 

It  also  attempted  to  identify  cases  where  the  importance  of 
a  high  level  of  technical  competence  might  be  situation 
dependent  and  to  identify  technical  areas  which  warrant 
increased  training  emphasis.  This  was  largely  an  explora¬ 
tory  study  which  will  hopefully  be  useful  as  a  basis  for 
ongoing  research. 
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Abstract 

This  study  explored  how  perceived  technical  compe¬ 
tence  of  Air  Force  acquisition  program  managers  is  related 
to  academic  education  and  intrinsic  technical  aptitude. 
Technical  competence  was  defined  as  the  program  manager's 
ability  to  assimilate  technical  program  information  and 
effectively  factor  it  into  his  decision  making  at  a  level 
commensurate  with  his  position.  This  study  also  attempted 
to  identify  certain  situational  factors  which  might 
increase  the  importance  of  a  program  manager  having  a  high 
level  of  technical  competence  and  to  identify  technical 
areas  which  warrant  increased  training  emphasis. 

Data  was  collected  through  personal  interviews 
with  officers  in  Air  Force  Specialty  Code  (AFSC)  2996  Pro¬ 
gram  Manager  (System  Program  Director)  positions  and  their 
technical  advisors.  Key  findings  were:  (1)  a  strong 
positive  correlation  exists  between  intrinsic  technical 
aptitude  and  perceived  technical  competence;  (2)  perceived 
technical  competence  of  officers  with  electrical  and  aero¬ 
nautical  engineering  degrees  was  significantly  higher  than 
that  of  officers  with  mathematics,  business,  or  liberal 
arts  degrees;  (3)  a  significant  proportion  of  the  officers 
interviewed  did  not  meet  current  mandatory  educational 
prerequisites  for  entry  into  the  AFSC  2996  specialty; 


(4)  officers  in  AFSC  2996  positions  who  function  as  single 
managers  for  developing  and  delivering  a  system  (i.e.,  true 
SPO  directors)  consider  technical  competence  to  be  a  sig¬ 
nificantly  more  important  job  requirement  than  do  other 
2996 's;  (5)  a  majority  of  those  interviewed  feel  technical 
ability  should  be  a  consideration  in  the  new  Acquisition 
Manager  Career  Development  Program  selection  process;  and 
(6)  computer  software  development  is  an  area  which  warrants 
increased  emphasis  in  the  training  of  future  acquisition 
program  managers. 

Among  the  recommendations  made  were  that  the  Air 
Force  consider  dividing  the  29XX  Program  Manager  utiliza¬ 
tion  field  into  separate  specialties  (one  for  those  heading 
system  program  offices  and  another  for  positions  with 
broader  oversight  responsibilities)  and  that  efforts  be 
made  to  match  individual  program  manager  attributes  with 
particular  job  needs  in  conjunction  with  or  in  addition  to 
the  new  Acquisition  Management  Career  Development  Program 
certification  and  selection  process. 


TECHNICAL  COMPETENCE  OF  AIR  FORCE 


ACQUISITION  PROGRAM  MANAGERS 


I.  Research  Problem 


Introduction 

The  System  Program  Office  (SPO)  director  of  an  Air 
Force  acquisition  program  holds  a  unique  program  manage¬ 
ment  position  (2:22).  As  the  single  manager  (by  regula¬ 
tion)  for  a  program,  the  SPO  director  is  generally  dele¬ 
gated  a  significant  anount  of  responsibility,  authority, 
and  accountability  for  attaining  program  objectives  (3:3, 
5-6) .  Successful  program  management  requires  the  SPO 
director  to  possess  a  wide  range  of  capabilities  and 
attributes  (7;  13) . 

Modern  systems  developed  and  acquired  by  the  Air 
Force  are  typically  associated  with  "high-technology." 

While  there  has  been  an  emphasis  on  career  development  and 
selection  policies  for  program  managers  recently,  a  review 
of  the  pertinent  regulations  indicates  that  the  emphasis 
has  been  on  management  training,  acquisition  experience, 
and  military  education  for  a  variety  of  acquisition  manage¬ 
ment  duties  leaving  an  ambiguous  policy  as  to  the  importance 
of  program  manager  technical  competence  (7;  12;  16:42,54, 


This  study  attempted  to  clarify  the  importance  of 
technical  competence  in  Air  Force  program  managers  and  to 
investigate  the  determinants  of  that  competence.  The  objec 
tive  was  to  identify  areas  where  changes  to  acquisition 
management  career  development  and  selection  policies  might 
be  warranted. 

Definitions 

The  meanings  of  key  terms  used  throughout  this 
report  are  as  follows: 

1.  Program  manager — an  Air  Force  officer  (lieuten¬ 
ant  colonel  or  colonel)  in  a  2996  or  2991  Duty  Air  Force 
Specialty  Code  (DAFSC)  position  as  defined  in  AFR  36-1. 
Program  managers  generally  provide  executive  management 
supervision  for  major  system  acquisition  programs. 

2.  SPO  director — a  program  manager  (defined  above) 
who  functions  as  the  single  manager  for  developing  and 
delivering  a  system.  The  SPO  director  organizes  and 
directs  a  system  program  office  (SPO)  and  is  the  same  as 
the  Program  Manager  (PM)  referred  to  in  AFR  800-2  and  the 
Packard  Commission  report  (3:2,5;  16:54). 

3.  SPO  director  candidate — an  Air  Force  officer 
(second  lieutenant  through  major)  covered  by  the  Air  Force 
Systems  Command  (AFSC)  Acquisition  Management  Career  Devel¬ 
opment  Program  (defined  in  AFSCR  36-5)  who  hopes  to  become 


a  SPO  director . 


4.  Technical — having  to  do  with  applied  science, 
engineering,  or  technology. 

5.  Technical  competence — the  ability  to  assimi¬ 
late  and  use  technical  information.  As  used  herein,  tech¬ 
nical  competence  applies  specifically  to  a  program  mana¬ 
ger's  ability  to  understand  technical  concepts  at  a  level 
of  detail  commensurate  with  his  management  position  and  to 
properly  factor  that  knowledge  into  his  decision  making. 
Technical  competence  is  assumed  to  be  a  function  of  educa¬ 
tion,  experience,  and  intrinsic  attributes. 

6.  Technical  aptitude — an  intrinsic  tendency  to 
consider  and  employ  technical  information.  A  natural  readi 
ness  to  learn  technical  knowledge.  Inquisitiveness  and  the 
propensity  to  ask  technical  questions  is  considered  a  fac¬ 
tor  of  technical  aptitude. 

7 .  Technical  undergraduate  degree — a  Bachelor  of 
Science  degree  with  specialization  in  engineering,  engineer 
ing  science,  engineering  management,  physical  science,  or 
math.  This  is  a  mandatory  prerequisite  for  becoming  a 
program  manager  (7) . 

8.  Technical  advisor — the  primary  individual 
charged  with  advising  a  program  manager  on  technical  pro¬ 
gram  issues  (e.g.,  the  Chief  Systems  Engineer). 

9.  Program  phase — a  stage  of  development  or  field¬ 
ing  of  a  system  acquired  by  the  Air  Force.  Concept  Defini¬ 
tion,  Demonstration  and  Validation,  Full-scale  Development, 


Production  and  Deployment,  and  Operations  Support  were  the 
phases  considered  in  this  study.  Acquisition  program 
managers  may  be  involved  in  any  of  these  phases,  but 
responsibility  for  management  of  a  system  often  is  trans¬ 
ferred  from  the  Air  Force  Systems  Command  SPO  director  to 
an  Air  Force  Logistics  Command  System  Manager  during  the 
Production  and  Deployment  phase. 

General  Issue 

Air  Force  policy  concerning  the  importance  of  pro¬ 
gram  manager  technical  competence  appears  to  be  ambiguous. 

On  one  hand,  technical  competence  would  seem  to  be  an 
important  element  of  effective  Air  Force  program  manage¬ 
ment  since,  by  regulation,  an  Air  Force  officer  must  have 
a  technical  undergraduate  degree  in  order  to  enter  the 
program  manager  2996  specialty  (7) .  On  the  other  hand,  it 
does  not  appear  that  a  serious  concern  actually  exists  for 
developing  and  certifying  technical  competence  in  program 
managers  since  the  Air  Force  Systems  Command  (AFSC)  Acquisi¬ 
tion  Management  Career  Development  Program  has  no  specific 
technology-oriented  requirements  in  its  certification 
process  and  does  not  include  technical  competence  in  its 
"best  qualified"  criteria  used  to  select  officers  eligible 
for  senior  program  management  positions  (12). 

The  technical  undergraduate  degree  requirement 
seems  to  be  reasonable  on  the  surface  because  a  SPO  director 


must  be  able  to  orchestrate  tradeoffs  between  technical 


options,  schedule,  and  cost  (3:3-6;  7;  19:234;  20:36). 
Technical  competence  may  be  important  in  effectively 
managing  technical  support  staff  and  in  discussing  tech¬ 
nical  issues  with  superiors  and  other  organizations 
(9:22-23).  However,  as  currently  delineated  in  the  exist¬ 
ing  policies,  the  technical  undergraduate  degree  is,  in 
effect,  a  "square"  to  be  filled  on  the  way  to  becoming  a 
program  manager.  Those  with  the  appropriate  type  of 
degree  are  apparently  deemed  to  be  sufficiently  "tech¬ 
nically  qualified."  No  further  evaluation  of  technical 
abilities  is  officially  involved. 

The  issue  at  hand  is  whether  or  not  these  policies 
are  reasonable  given  the  current  emphasis  on  grooming  and 
selecting  the  highest  quality  acquisition  managers  possible 
(12;  16:65-66) . 

Specific  Issues 

Within  the  general  issue  described  above  are  a 
number  of  issues  which  this  study  addressed  specifically. 

Benefits  of  a  Technical  Degree.  A  Icey  issue  inves¬ 
tigated  in  this  study  was  the  link  between  academic  educa¬ 
tional  background  and  perceived  technical  competence  in 
program  managers.  In  defining  the  qualifications  neces¬ 
sary  to  become  a  program  manager,  AFR  36-1  states  that 


5 


c-r.  ■ 


.  .  .  undergraduate  academic  specialization  in  engineer¬ 
ing,  engineering  science,  engineering  management, 
physical  science,  or  math  is  mandatory  for  entry  into 
the  specialty  [7] . 

The  Acquisition  Management  Career  Development  Program 
requires  a  "Master's  Degree  (or  higher)  in  a  technical  or 
management  field  appropriate  to  program  management"  for 
"Level  III"  or  "Level  IV"  certification  which  would  apply 
to  program  managers  (12) . 

If  the  purpose  of  a  technical  academic  education  is 
to  provide  an  individual  with  specific  technical  knowledge 
to  be  used  in  performing  SPO  director  duties,  these  require¬ 
ments  seem  almost  too  broad.  One  might  ask  whether  a  math 
major  has  a  satisfactory  understanding  of  electronics 
principles  to  manage  effectively  in  a  service  hungry  for 
electrical  engineers  (15)  .  Contrarily,  given  the  wide 
range  of  technical  issues  the  SPO  director  of  a  highly  tech¬ 
nical  program  is  likely  to  encounter  and  the  narrow  focus 
of  specialties  within  technical  degrees  (e.g.,  optics, 
electromagnetic  fields/waves,  and  digital  processing  are 
separable  specialties  within  an  electrical  engineering 
degree  program) ,  it  seems  unreasonable  to  expect  academic 
education  to  provide  specific  background  knowledge  in  all 
areas  likely  to  be  encountered,  particularly  in  light  of 
the  rate  at  which  technology  is  advancing.  Nevertheless, 
the  ever-increasing  reliance  on  electronics  and  computers 
in  modern  systems  might  serve  to  justify  a  move  toward 
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more  stringent  academic  qualifications  such  as  requiring 
some  level  of  SPO  director  technical  competence  in  these 
particular  areas. 

The  rationale  behind  requiring  the  stated  range 
of  acceptable  academic  specialties  might  be  the  belief 
that  all  of  the  specialties  foster  a  desired  way  of  think¬ 
ing  (e.g.,  quantitative  problem  solving)  and  cover  some 
desirable  set  of  core  areas  such  as  calculus  and  basic 
computer  programming  which  provide  the  necessary  back¬ 
ground  education.  Along  with  this  is  the  notion  that 
these  undergraduate  programs  provide  a  "quality  control" 
function.  An  individual  must  be  somewhat  intelligent  and 
capable  in  order  to  obtain  a  degree  in  those  specialties. 

Not  everyone  would  agree  that  a  technical  degree 
is  desirable  for  program  managers,  however.  Some  work  has 
been  done  to  show  that  a  nontechnical,  "generalist"  back¬ 
ground  is  preferred  for  senior  military  managers  (18) . 

A  basic  assumption  of  this  portion  of  the  study 
was  that  if  a  technical  academic  education  is  of  benefit 
to  a  program  manager,  that  benefit  would  be  manifested  in 
an  increased  ability  to  assimilate  and  use  technical  infor¬ 
mation  and  to  factor  that  knowledge  into  decision  making 
(i.e.,  increased  technical  competence).  The  issue  addressed 
in  this  study  was  whether  or  not  the  type  of  undergraduate 
or  graduate  degree  held  by  a  program  manager  could  be 


shown  to  be  associated  with  his  or  her  ability  to  deal 
appropriately  with  technical  program  issues. 

It  should  be  noted  that,  since  this  was  ex  post 
facto  research  versus  an  experiment,  if  a  relationship  was 
shown  to  exist  between  type  of  academic  education  and  tech 
nical  competence,  it  would  not  necessarily  imply  causality 


The  Role  of  Intrinsic  Technical  Aotitude.  The 


second  issue  this  study  dealt  with,  taken  as  somewhat  of 
a  converse  to  the  first  issue  discussed  above,  was  the 
relationship  between  intrinsic  technical  aptitude  and  tech¬ 
nical  competence.  Aptitude  is  defined  by  Webster  as: 

1.  Readiness  for  learning;  aptness. 

2.  General  fitness  or  suitability;  appropriate¬ 
ness. 

3.  Natural  disposition  or  tendency  to  a  particular 
action  or  effect. 

4.  Natural  or  acquired  capacity  or  ability. 

5.  Potential  as  distinguished  for  developed 
ability;  capacity  for  learning  a  certain  performance 
or  kind  of  work  [22:1351. 

The  third  and  fifth  definitions  come  closest  to  the  con¬ 
struct  considered  here.  A  program  manager's  natural 
tendency  to  personally  engage  technical  issues,  measured 
in  this  study  by  propensity  to  ask  questions  on  technical 
aspects  of  the  program,  was  part  of  the  technical  aptitude 
construct.  The  second  part  of  the  construct  was  potential 
for  understanding  newly  presented  technical  information, 
which  was  indicated  by  quickness  in  learning  new  (not  pre¬ 
viously  familiar)  technical  information. 


The  type  of  undergraduate  or  graduate  degree  an 


individual  holds  may  not  necessarily  be  a  valid  indication 
of  his  abilities  or  inclinations  (8) .  Assuming  that  a  pro¬ 
gram  manager  must  deal  with  a  wide  variety  of  technical 
issues  and  that  much  of  the  technical  information  he  gains 
is  from  other  than  formal  academic  education,  technical 
aptitude  could  conceivably  play  a  greater  role  in  attain¬ 
ing  a  high  degree  of  technical  competence  than  formal  train¬ 
ing  or  education.  At  issue  here  was  whether  intrinsic 
technical  aptitude,  aside  from  formal  education  or  experi¬ 
ence,  is  significantly  and  positively  related  to  tech¬ 
nical  competence. 

Program  Phase  as  a  Moderator.  Perhaps  the  charac¬ 
teristics  associated  with  different  programs  call  for  dif¬ 
ferent  technical  competency  levels  in  the  program  managers 
who  manage  them.  Conceivably  a  number  of  program-related 
moderating  variables  could  exist  which  would  correspond  not 
only  with  the  level  of  technical  competence  appopriate  for 
managing  a  particular  program,  but  to  specific  technical 
specialty  areas  as  well.  For  example,  a  high  level  of 
program  manager  technical  competence  may  be  of  greater 
importance  on  programs  involving  very  advanced  technology 
(2:36).  Other  research  has  indicated  that  the  intensity 
of  management  conflicts  associated  with  technical  issues 
varies  significantly  during  different  phases  of  a  program's 


life  cycle  (20) .  This  study  explored  the  notion  that  pro¬ 
gram  manager  technical  competence  may  be  a  more  important 
factor  when  systems  are  in  certain  program  phases. 

Relative  Importance  of  Technical  Competence.  SPO 
directors  need  an  extremely  broad  range  of  skills,  knowl¬ 
edge,  and  experience  (12;  13) .  The  importance  of  tech¬ 
nical  competence  relative  to  other  factors  such  as  manage¬ 
ment  ability,  leadership,  and  operational  experience  is  a 
logical  concern.  Would  a  lack  of  technical  competence  on 
the  part  of  a  program  manager  actually  be  detrimental  to  a 
program  or  is  a  high  level  of  technical  competence  simply 
a  "nice  to  have"  attribute?  Do  program  managers  believe 
that  technical  competence  is  important  enough,  relative  to 
other  criteria,  to  be  a  consideration  in  selecting  future 
program  managers? 

Research  Questions 

The  following  research  questions  are  based  on  the 
issues  discussed  above  and  are  formulated  for  statistical 
analysis. 

Research  Question  1.  is  program  manager  technical 


competence  associated  with  type  of  academic  educational 
background? 


Research  Question  2. 


Is  program  manager  technical 


competence  related  to  intrinsic  technical  aptitude? 

Research  Question  3.  Does  the  perceived  impor¬ 
tance  of  a  high  level  of  program  manager  technical  compe¬ 
tence  depend  upon  the  acquisition  phase  of  the  program 
being  managed? 

Research  Question  4 .  According  to  program  mana¬ 
gers,  should  technical  competence  be  included  in  the 
criteria  used  to  select  "best  qualified"  officers  eligible 
to  fill  key  middle  management  and  senior  program  manage¬ 
ment  positions  under  the  new  Acquisition  Management  Career 
Development  Program? 

Variables  and  Relationships 

The  following  variables  and  relationships  apply  to 
the  research  questions.  Operationalization  and  measurement 
of  each  variable  is  described  in  detail  in  Chapter  III. 

Research  Question  1.  This  question  concerned  the 
relationship  between  type  of  academic  educational  degree 
and  perceived  program  manager  technical  competence.  One 
independent  and  one  dependent  variable  were  involved.  The 
independent  variable  was  the  program  manager's  academic 
educational  background  which  was  treated  as  a  categorical 
(nominal)  variable.  Specific  types  of  academic  degrees 
were  placed  into  four  different  categories  based  on  how 


closely  courses  in  the  various  curricula  were  believed  to 
apply  directly  to  current  technical  issues  likely  to  be 
encountered  by  program  managers.  The  categories  were: 

1.  Electrical,  computer,  and  aerospace  engi¬ 
neering  . 

2.  Other  engineering,  physical  sciences,  math 
and  computer  science . 

3.  Business/management,  and  other  B.S./M.S. 
degrees  not  included  in  the  above  categories. 

4.  All  other  nontechnical  (e.g.,  B.A.)  degrees. 

The  dependent  variable  (interval  level)  was  the 

perceived  level  of  program  manager  technical  competence 
which  was  measured  using  a  subset  of  items  from  the  Wagner 
and  Morse  sense  of  competence  instrument  (21)  .  Five  items 
from  this  instrument  were  tailored  to  apply  specifically 
to  technical  competence  and  used  to  compute  a  composite 
variable  which  constituted  the  dependent  variable.  The 
items  used  addressed  the  following: 

1.  How  well  the  program  manager's  level  of  tech¬ 
nical  competence  met  expectations  for  doing  his  job. 

2.  How  well  the  technical  competence  exhibited  by 
the  program  manager  might  serve  as  a  model  for  an  appren¬ 
tice  . 

3.  To  what  degree  the  program  manager's  talents 
lay  in  areas  other  than  in  dealing  with  technical  aspects 
of  programs  he  supervised. 


4.  How  well  the  program  manager  familiarized  him¬ 
self  with  technical  program  issues  considering  the  amount 
of  time  in  the  job . 

5.  To  what  degree  the  program  manager  had  all  of 
the  technical  skills  necessary  to  perform  well  in  his  job. 

Research  Question  2 .  This  question  addressed  the 
relationship  between  technical  aptitude  and  technical 
competence.  It  involved  the  relationship  between  two  vari¬ 
ables.  The  independent  variable  (interval  level)  was 
perceived  program  manager  technical  aptitude.  For  the 
purpose  of  this  study,  the  construct  of  technical  aptitude 
included  the  following  characteristics: 

1.  Program  manager  inquisitiveness  on  technical 
program  aspects. 

2.  Program  manager  quickness  in  assimilating  new 
technical  information. 

The  dependent  variable  (interval),  program  manager  tech¬ 
nical  competence,  was  the  same  as  for  Research  Question  1 
above . 

Research  Question  3.  This  question  addressed  the 
relationship  between  the  importance  of  program  manager 
technical  competence  and  the  program  phase  of  the  system 
being  acquired.  It  involved  the  relationship  between  two 
variables.  The  independent  variable  was  program  phase 


category  (a  nominal  variable) .  The  following  categories 
were  used: 

1.  Concept  definition 

2.  Demonstration  and  Validation 

3 .  Full-scale  Development 

4 .  Production  and  Deployment 

5.  Operations  Support 

Values  of  the  program  phase  variable  were  based  on  the 
acquisition  phase  program  managers  indicated  their  pro¬ 
grams  were  in  at  the  time  of  data  collection.  In  cases 
where  a  system  was  in  more  than  one  program  phase  (e.g., 
F-16  aircraft  are  operational,  in  production,  and  under¬ 
going  continued  development) ,  program  phase  was  considered 
that  which  the  program  manager  felt  posed  the  most  impor¬ 
tant  technical  issues. 

The  dependent  variable  (interval)  was  the  perceived 
importance  of  the  program  manager  having  a  high  level  of 
technical  competence  to  perform  well  in  his  particular  job 
Two  survey  items  were  used  to  measure  this  construct.  One 
item  asked  how  important  a  consideration  technical  com¬ 
petence  should  be  in  selecting  a  hypothetical  replacement 
for  the  program  manager.  The  other,  a  reverse  scored  item 
asked  whether  a  lack  of  technical  competence  on  the  part 
of  a  new  program  manager  could  be  a  serious  detriment  to 
the  program. 


Research  Question  4 .  This  question  addressed 
whether  technical  competence  is  important  enough  to  be 
a  separate  criterion  (at  the  same  level  as  duty  perform¬ 
ance,  leadership  ability,  and  operational  experience)  in 
selecting  Air  Force  officers  for  high  level  acquisition 
management  positions.  It  involved  a  single  variable, 
opinion  as  to  whether  or  not  technical  competence  was 
sufficiently  important,  which  was  treated  as  a  Bernoulli 
variable  (yes  or  no) . 

Criteria  for  Data  Source 

The  data  source  for  this  study  was  comprised  of 
two  groups:  individuals  filling  Air  Force  Specialty  Code 
(AFSC)  2996  Program  Manager  (Systems  Program  Director)  posi¬ 
tions  (termed  program  managers  in  this  study)  and  their 
technical  advisors.  Rather  than  using  all  officers  who 
themselves  hold  or  had  held  primary,  secondary,  or  tertiary 
2996  AFSCs  as  the  population,  those  actually  filling  2996 
positions  were  used  in  an  effort  tc  tie  the  data  collected 
directly  to  duties  being  performed.  This  allowed  for  data 
to  be  collected  from  some  individuals  without  technical 
undergraduate  degrees,  and  therefore  not  meeting  all  2996 
requirements,  yet  who  were  filling  program  manager  positions 


The  technical  advisors  included  in  the  data  source 
were  those  wnom  program  managers  indicated  were  primary  in 
advising  them  on  technical  program  issues.  The  reason  for 


obtaining  opinion  data  from  both  program  managers  and  their 
technical  advisors  was  to  provide  a  check  for  convergent 
validity  on  the  ratings  of  program  manager  technical  com¬ 
petence  and  technical  aptitude. 

Research  Approach 

The  approach  taken  in  this  research  effort  was  to 
conduct  personal  interviews  with  program  managers  and  their 
technical  advisors.  The  data  collected  during  these  inter¬ 
views  included  such  things  as  program  manager  educational 
background,  program  phase,  and  ratings  of  program  manager 
technical  competence  and  technical  aptitude. 

Because  the  question  of  whether  program  managers 
must  be  "engineers"  is  potentially  subject  to  personal 
biases  and  is  apparently  an  open  issue  (13:24)  and  because 
there  is  perhaps  a  natural  tendency  for  a  person  to  rate 
himself  "very  competent"  in  any  job-related  dimension, 
several  things  were  done  in  the  approach  taken  on  this 
study.  First,  the  interview  survey  instrument  was  care¬ 
fully  constructed  to  measure  specific  job- related  percep¬ 
tions  and  behaviors  rather  than  simply  asking  broad  ques¬ 
tions  of  opinion  like  "Do  you  think  a  program  manager's 
educational  degree  makes  a  difference?"  Second,  use  of 
personal  interviews  rather  than  mail  surveys  helped  to 
ensure  that  respondents  fully  understood  the  questions 
asked  and  the  terms  used.  Third,  ratings  by  technical 
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Based  on  information  gleaned  from  primary  and 
secondary  sources,  this  section  provides  background  on 
Air  Force  program  manager  responsibilities  and  technical 
qualifications,  why  technical  competence  may  be  an  impor¬ 
tant  consideration,  and  measurement  of  technical  compe¬ 
tence  . 

Air  Force  Acquisition  Managers 

The  Air  Force  classifies  the  types  of  duty  offi¬ 
cers  perform  according  to  "utilization  fields"  and  "special¬ 
ties."  Each  type  of  job  is  assigned  a  four-digit  Air  Force 
Specialty  Code  (AFSC) .  The  first  two  digits  of  an  AFSC 
define  a  utilization  field.  Addition  of  the  third  digit 
defines  a  specialty.  The  fourth  digit  corresponds  to  the 
highest  grade  an  officer  in  a  particular  specialty  may  hold. 
Air  Force  Regulation  (AFR)  36-1  describes  the  officer 
specialty  classification  system  and  provides  attachments 
which  summarize  the  duties,  responsibilities,  and  qualifi¬ 
cations  pertaining  to  each  specialty. 

A  number  of  different  utilization  fields,  such 
as  Scientific  (26XX) ,  Development  Engineering  (28XX) ,  and 
Logistics  Plans  and  Programs  (66XX)  to  name  a  few,  are 
associated  with  acquisition  management.  The  term 
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acquisition  manager  generally  applies  to  officers  in  the 
Acquisition  Program  Management  (27XX)  and  Program  Manage¬ 
ment  (29XX)  utilization  fields  (12:1).  The  three  officer 
specialties  within  these  utilization  fields  are: 

1.  AFSC  2724  Acquisition  Project  Officer 

2.  AFSC  2716  Acquisition  Management  Officer 

3.  AFSC  2996  Program  Manager  (Systems  Program 
Director) 

Officers  in  AFSC  2724  positions  may  range  in  grade 
from  second  lieutenant  through  major.  They  "assist  in 
planning  and  managing  system,  subsystem,  or  equipment 
acquisition  programs"  and  often  worlc  for  program  managers 
in  system  program  offices  (SPOs)  (6) . 

Officers  in  AFSC  2716  positions  are  the  next  level 
in  terms  of  rank  and  responsibility.  An  Acquisition  Man¬ 
agement  Officer  "performs  as  Program  Manager  (PM)  for  the 
acquisition  of  any  program  not  meeting  the  definition  of 
a  major  program."  The  grade  spread  for  2716 's  is  major 
through  colonel.  Those  in  AFSC  2716  positions  generally 
do  not  head  SPOs  (5) . 

AFSC  2996  positions  are  filled  by  lieutenant 
colonels  or  colonels.  The  2996  program  manager  "directs 
and  provides  executive  management  supervision  for  major 
acquisition  programs."  SPOs  are  generally  headed  by 
officers  in  AFSC  2996  positions  (7). 


This  study  focused  on  officers  in  2996  positions 
and,  as  used  herein,  the  term  "program  manager"  refers  to 
these  officers.  The  term  "SPO  director,"  as  used  in  this 
report,  refers  to  the  subset  of  program  managers  who  head 
SPOs  and  direct  acquisition  activities  of  a  single  major 
system.  In  many  cases,  those  in  AFSC  2996  positions  "pro¬ 
vide  executive  management  supervision"  at  organizational 
levels  that  are  administratively  higher  than  those  of  SPO 
directors . 

The  President's  Blue  Ribbon  Commission  on  Defense 
Management  (the  Packard  Commission)  in  its  final  report, 
recommended  giving  "acquisition  personnel  more  authority 
to  do  their  jobs,"  reducing  the  number  of  layers  of  super¬ 
vision,  and  establishing  direct  mechanisms  for  program 
managers  of  major  programs  to  report  directly  to  high 
ranking  officials  in  newly  created  Program  Executive  Offi¬ 
cer  (PEO)  positions  on  program- related  matters  (16:61,73). 
This  recommendation  could  be  interpreted  as  a  call  to 
increase  the  autonomy  and  authority  of  SPO  directors. 

The  Acquisition  Management 
Career  Development  Program 

The  Acquisition  Management  Career  Development 
Program  (AMCDP)  is  governed  by  Air  Force  Systems  Command 
Regulation  (AFSCR)  36-5  (12) .  While  AFSCR  36-5  had  not 
been  officially  signed  and  dated  at  the  time  this  study 


was  conducted,  the  AMCDP  was  experiencing  de  facto  imple¬ 
mentation  by  Air  Force  Systems  Command  (AFSC)  and  the  Mili¬ 
tary  Personnel  Center  (MPC) . 


The  primary  focus  of  the  AMCDP  is  on  officers  per¬ 
forming  duty  in  the  26XX  through  29XX  utilization  fields. 

It  provides  "a  phased  professional  certification  process" 
which  "contributes  to  the  motivation  of  individuals  toward 
career  development  efforts"  and  "a  formal  selection  pro¬ 
cess"  intended  to  "identify  those  officers  best  qualified 
to  assume  senior  acquisition  management  responsibilities" 
(12:5)  . 

In  the  certification  process,  "specific  quali¬ 
fication  requirements  are  identified  for  academic  education, 
specialty  training,  professional  military  education,  and 
acquisition  related  experience"  (12:4).  Four  sequential 
certification  levels  are  specified,  each  associated  with 
fulfillment  of  a  minimum  set  of  requirements  in  the  above- 
mentioned  areas.  The  academic  education  requirements  for 
the  highest  certification  level  are  any  Bachelor's  Degree 
and  a  "Master's  Degree  (or  higher)  in  a  technical  or  manage¬ 
ment  field  appropriate  to  program  management"  (12:13). 
Specialty  training  applies  to  courses  such  as  Air  Force 
Institute  of  Technology  (AFIT)  SYS  400  (Intermediate  Pro¬ 
gram  Management)  and  the  Defense  Systems  Management  College 
(DSMC)  Program  Management  Course  (12:12-13).  Squadron 
Officer  School,  Intermediate  Service  School,  and  Senior 


Service  School  fulfill  the  professional  military  education 
requirements.  A  mix  of  SPO,  operational,  headquarters, 
and  other  acquisition-related  duties  are  considered  for 
the  experience  requirements  (12). 

The  selection  process  of  the  AMCDP  involves  two 

lists--an  Acquisition  Managers  List  (AML) ,  and  a  Senior 

Acquisition  Managers  List  (SAML) .  The  AML  would  establish 

"a  pool  of  officers  who  are  qualified  to  fill  )cey  middle 

management  positions."  The  SAML  would  be  comprised  of 

.  .  .  officers  qualified  to  assume  senior  program  man¬ 
agement  positions,  including  Selected  Acquisition 
Review  (SAR)  and  Air  Force  Systems  Acquisition  Review 
Council  (AFSARC)  program  manager  responsibilities. 

Each  year,  a  selection  board  would  choose  about  100  officers 
for  the  AML  and  50  officers  for  the  SAML  on  a  "best  quali¬ 
fied"  basis.  For  SAML  selection,  "duty  performance,  demon¬ 
strated  leadership  ability,  and  operational  experience" 
would  be  "weighed  heavily."  The  AML  grade  requirement 
would  include  majors  "with  enough  time-in-grade  as  to  be 
eligible  for  secondary  zone  promotion  to  lieutenant  colonel 
SAML  officers  would  need  to  "be  a  lieutenant  colonel 
selectee  or  above"  (12:17,19). 
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technical  solutions  and  tradeoffs,  assess  risks,  and  weigh 
technical  issues  against  cost  and  schedule  considerations 
(19:233-235;  20:36).  In  addition,  technical  competence 
can  play  an  important  role  in  a  SPO  director's  effective¬ 
ness  in  interpersonal  relationships  (9:16).  Without  an 
understanding  of  the  technology  at  hand,  a  SPO  director 
may  not  be  able  to  win  the  confidence  of  SPO  team  members 
or  to  build  credibility  with  higher  command  levels  (19:235). 

The  problems  faced  by  a  project  manager  with  a 
lack  of  technical  expertise  are  that  he  may  lose  con¬ 
trol  of  a  project  and  cause  resentment  on  the  part 
of  participants  who  feel  that  he  does  not  understand 
their  position.  A  lack  of  technical  expertise  may 
also  adversely  affect  the  decision-making  process. 

The  project  manager  must  either  rely  on  team  members 
for  technical  decisions  or  delay  the  decision  until 
he  can  consult  a  third  party  (9:22-23). 

On  the  other  hand,  technical  competence  alone  is 
certainly  not  sufficient  for  success  in  management.  A  sig¬ 
nificant  degree  of  management  training  and  aptitude  is 
needed  as  well  (1:39-40).  Also,  there  is  some  concern  that 
a  SPO  director  who  is  overly  competent  technically  may  con¬ 
centrate  too  much  on  technical  details  at  the  expense  of 
other  program  activities  (9:22-23).  Indeed,  there  is  a 
belief  that  those  who  are  highly  skilled  technically  may 
not  have  the  "generalist"  background  desired  for  senior 
military  management  positions  (18:8). 

Academic  education  for  senior  military  managers 
(technical  versus  liberal  arts/humanities  degrees)  is  an 
area  of  some  debate.  A  common  presumption  exists  that  the 
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ideal  educational  background  for  managers  of  technical  pro¬ 
grams  is  an  undergraduate  degree  in  engineering  or  the 
physical  sciences  followed  by  masters  level  study  in  busi¬ 
ness  administration  (10:94).  This  presumption  is  not 
unchallenged,  though. 

According  to  Dr.  William  Snyder,  current  Air  Force 
policy  is  oriented  toward  commissioning  new  officers  with 
technical  undergraduate  degrees.  This  is  evidenced  by  the 
fact  that  Air  Force  ROTC  scholarships  are  only  awarded  to 
those  with  engineering  and  technical  majors  (18:7-8). 

While  this  enables  junior  officers  to  make  productive  con¬ 
tributions  in  early  assignments,  he  believes  that  those 
with  more  general,  nontechnical  training  will  be  better 
suited  for  executive-level  management  jobs.  To  support 
his  point,  he  cites  results  of  various  studies  which  indi¬ 
cate  that  those  with  liberal  arts  degrees  have  the  advan¬ 
tage  in  attaining  the  highest  executive  levels  in  business 
(18:8-12)  . 

Other  studies  indicate  a  lack  of  consensus  on  the 
need  for  SPO  directors  to  have  technical  backgrounds. 
Patricia  Kelly,  a  professor  at  the  Defense  Systems  Manage¬ 
ment  College  (DSMC) ,  conducted  a  survey  of  government  and 
industry  program  managers  for  her  article  "Searching  for 
Excellence  in  the  Program  Office."  One  of  the  topics  she 
addressed  was  the  need  for  military  program  managers  to 
have  technical  backgrounds.  She  reported  that  twelve 


respondents  said  that  a  program  manager  "must  be  an  engi¬ 
neer,"  must  "have  a  technical  background,"  or  must  "be 
technically  qualified."  There  were  three  people  who  said 
a  technical  background  was  unnecessary.  Interestingly, 
two  of  the  three  respondents  who  indicated  that  a  technical 
background  was  not  important  were  general  officers  while 
most  of  those  saying  it  was  important  were  engineers 
(13:23-24)  . 

Other  factors  work  to  further  complicate  the  ques¬ 
tion  of  academic  educational  background  and  the  underlying 
importance  of  technical  competence  in  SPO  directors. 

First,  the  widely  held  assumption  that  a  technical  degree 
makes  one  technically  competent  may  not  be  valid,  especi¬ 
ally  given  the  pace  at  which  technology  is  advancing  and  the 
limitations  on  what  is  actually  covered  in  any  particular 
engineering  curriculum  (8).  Secondly,  the  importance  of 
SPO  director  technical  competence  may  depend  on  the  nature 
of  the  program  itself.  More  highly  technical  programs  may 
call  for  a  higher  level  of  technical  competence  on  the 
part  of  the  SPO  director  (2:36;  9:19).  Also,  there  is 

some  evidence  that  program  phase  may  be  a  factor  to  con¬ 
sider.  Technical  issues  may  be  of  greater  concern  during 
actual  system  design  and  development  than  in  earlier 
planning  or  later  fielding  phases  (20:39).  Unique  charac¬ 
teristics  of  particular  jobs  are  likely  to  be  factors  as 
well . 
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Air  Force  Program  Manager 
Technical  Requirements 


For  the  AFSC  2996  Program  Manager  Specialty, 

AFR  36-1  states  that  "undergraduate  academic  specializa¬ 
tion  in  engineering,  engineering  science,  engineering  man¬ 
agement,  physical  science,  or  math  is  mandatory  for  entry" 
(7).  Requirements  for  AFSC  2724  and  2716  specialties  are 
the  same  except  that  "business/management"  specialization 
is  also  acceptable  (5;  6)  . 

A  Master's  Degree  in  a  technical  field  "appropri¬ 
ate  to  program  management"  satisfies  academic  education 
requirements  for  the  highest  certification  level  of  the 
AMCDP,  but  a  management  degree  will  do  so  as  well  (12:13)  . 
AFSCR  36-5  does  not  place  any  limitations  on  undergraduate 
education,  but  does  require  two  "acquisition-related" 
specialty  courses  for  certification  at  the  second  certifi¬ 
cation  level,  some  of  which  are  technically  oriented  (11; 
12:11-12) . 

Measurement  of  Technical  Competence 

This  research  effort  required  that  ratings  of  tech¬ 
nical  competence  be  obtained  for  fairly  high  level  program 
managers.  Ta)cing  an  objective  measure  of  this  character¬ 
istic  would  obviously  be  subject  to  political  and  practical 
pitfalls.  One  might  humorously  imagine  the  pi  Lght  of  a 
young  graduate  student  seeking  approval  for  and  actually 
attempting  to  administer  some  sort  of  proficiency  test  to 


senior  Air  Force  colonels.  A  measurement  method  based  on 
opinions  or  perceptions  has  obvious  practical  advantages 
(23:4)  . 

First  lieutenant  Benjamin  Wilson  conducted  a  rather 
extensive  review  of  literature  pertaining  to  "sense  of 
competence"  measurement  for  his  Master  of  Science  thesis 
in  1985.  He  used  a  survey  instrument  developed  by  Wagner 
and  Morse  to  measure  sense  of  competence  of  Air  Force 
junior  officers  in  Civil  Engineering  (CE)  jobs.  By 
tailoring  of  the  wording  in  the  instrument,  Wilson  obtained 
ratings  from  the  CE  officers  and  their  supervisors  on  over¬ 
all  sense  of  job  competence,  sense  of  "technical"  compe¬ 
tence,  sense  of  managerial  competence,  and  job  knowledge 
(23:26-34) . 

The  instrument  developed  by  Wagner  and  Morse  was 
a  23-item  "valid,  reliable  paper-and-pencil  questionnaire." 
A  strong  relationship  between  the  sense  of  competence  index 
measured  with  this  instrument  and  actual  job  performance 
and  effectiveness  was  shown  in  separate  tests  conducted 
with  employees  from  an  aerospace  company  and  from  county 
government  departments  (21;  23:29). 

Snyder  and  Morris  conducted  research  which  identi¬ 
fied  15  of  the  23  Wagner  and  Morse  instrument  items  that 
had  improved  reliability  "across  different  samples  and 
settings"  (17;  23:29). 


III.  Method 


This  section  describes  the  methods  used  to  collect 
and  analyze  data  obtained  in  support  of  this  research 
project . 

Justif ication 

A  survey  approach  was  chosen  for  this  research  pri¬ 
marily  because  there  was  no  data  otherwise  found  to  be 
available  for  answering  the  research  questions;  data  had  to 
be  collected.  Experimentation  was  clearly  not  an  option 
as  control  of  individuals'  education  and  careers  would  have 
been  required.  Objective  measurement  of  technical  compe¬ 
tence  and  technical  aptitude  was  deemed  to  be  impractical 
and  inappropriate  under  the  constraints  of  this  effort, 
as  was  direct  observation  by  the  researcher,  leaving  survey 
measurement  of  perceptions  and  opinions  as  the  only  reason¬ 
able  data  collection  alternative.  Because  of  the  desire  to 
use  technical  advisor  ratings  matched  with  program  manager 
self-ratings  (for  purposes  of  validity) ,  and  to  avoid 
potential  misunderstanding  of  the  terms  and  constructs  used 
personal  interviews  were  chosen  over  mail  surveys  (ques¬ 
tionnaires)  as  the  preferred  survey  approach. 


3.  Ratings  of  the  perceived  importance  of  a  pro¬ 
gram  manager  having  a  high  level  of  technical  competence 
differ  significantly  depending  upon  the  program  phase  of 
the  system  being  acquired. 

4.  A  majority  (significantly  more  than  50  percent) 
of  those  in  program  manager  positions  and  their  technical 
advisors  feel  that  technical  competence  should  be  part  of 
the  criteria  used  to  select  officers  for  key  middle  manage¬ 
ment  and  senior  program  management  positions. 

Hypothesis  1.  This  hypothesis  was  used  to  test 
the  relationship  between  educational  background  (under¬ 
graduate  or  graduate  specialty)  and  perceived  technical 
competence  in  program  managers.  The  null  hypothesis  was 
that  technical  competence  ratings  of  program  managers  would 
not  be  significantly  different  for  individuals  with  differ¬ 
ent  educational  backgrounds. 

Educational  background  (the  independent  variable) 
was  treated  as  a  nominal  variable.  Specific  types  of  aca¬ 
demic  degrees  were  categorized  into  four  different  cate¬ 
gories  based  on  how  closely  courses  in  the  various  cur¬ 
ricula  were  believed  to  apply  directly  to  current  technical 
issues  likely  to  be  encountered  by  program  managers.  The 
four  categories  were: 

1.  Category  A — electrical,  computer,  and  aero¬ 
space  engineering. 
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2.  Category  B — other  engineering,  physical 
sciences,  math  and  computer  science. 

3.  Category  C — business/management  and  other 
B.S./M.S.  degrees  not  included  in  categories  A  or  B . 

4.  Category  D — all  other  nontechnical  (e.g.,  B.A.) 

degrees . 

The  dependent  variable  was  program  manager  tech¬ 
nical  competence.  Both  program  manager  self-ratings  and 
technical  advisor  ratings  of  program  manager  technical 
competence  were  to  be  obtained.  Perceived  program  manager 
technical  competence  was  measured  using  questions  from  the 
Wagner  and  Morse  sense  of  competence  instrument  (22) . 

Snyder  and  Morris  had  identified  15  items  from  the  original 
23-item  Wagner  and  Morse  instrument  which  offered  con¬ 
sistent  reliability  loadings  "across  different  samples  and 
settings"  (17) .  Five  items  from  the  Snyder  and  Morris  sub¬ 
set  were  chosen  and  tailored  to  apply  specifically  to  tech¬ 
nical  competence  in  an  acquisition  management  setting. 
Ratings  from  these  five  items  were  used  to  compute  a  com¬ 
posite  variable  which  constituted  the  dependent  variable 
(23:30).  Responses  to  each  of  the  five  items  were  rated 
on  a  five-point  scale.  The  sum  of  the  five  individual 
ratings  was  taken  as  the  value  of  the  composite  variable 
which  was  termed  a  technical  competence  index. 


Both  the  program  manager  and  technical  advisor 
survey  instruments  included  the  following  five  items  con¬ 
cerning  perceptions  of  the  program  manager's  technical  com¬ 
petence  : 

1.  How  well  the  program  manager's  level  of  tech¬ 
nical  competence  met  expectations  for  doing  his  job. 

2 .  How  well  the  technical  competence  exhibited  by 
the  program  manager  might  serve  as  a  role  model  for  an 
apprentice. 

3.  To  what  degree  the  program  manager's  talents 
lay  in  areas  other  than  in  dealing  with  technical  aspects 
of  programs  he  supervised. 

4.  How  well  the  program  manager  had  "come  up  to 
speed"  on  technical  program  issues  considering  the  amount 
of  time  in  the  job. 

5.  To  what  degree  the  program  manager  had  all  the 
technical  skills  necessary  to  perform  well  in  his  job. 

Statistical  Testing.  Unbalanced  single¬ 
factor  analysis  of  variance  (ANOVA)  at  a  .01  level  alpha 
was  used  to  determine  if  a  significant  difference  existed 
in  values  of  the  dependent  variable  between  two  or  more 
educational  categories.  The  ANOVA  F-test  indicates  whether 
a  significant  difference  exists  between  at  least  two  cate¬ 
gories,  but  does  not  identify  where  the  differences  occur. 
The  Bonferroni  multicomparison  technique  was  then  used  to 


identify  between  which  categories  differences  existea. 
Bonferroni  was  used  because  the  problem  was  unbalanced. 
Since  the  Bonferroni  technique  controls  Type  I  error  well, 
but  does  not  control  Type  II  errors  as  well  as  other  tech¬ 
niques,  meaning  some  significant  differences  could  be 
masked,  a  fairly  large  (e.g.,  .1)  level  alpha  was  used. 

The  testing  was  to  have  been  done  twice;  once  using 
the  program  manager  self-ratings  and  once  using  the  tech¬ 
nical  advisor  ratings  of  program  manager  technical  com¬ 
petence  . 

Hypothesis  2.  This  hypothesis  was  used  to  test 
the  relationship  between  perceived  technical  aptitude  and 
technical  competence  of  program  managers.  The  null  hypo¬ 
thesis  was  that  there  is  not  a  significant  positive  corre¬ 
lation  between  ratings  of  program  manager  technical  apti¬ 
tude  and  ratings  of  program  manager  technical  competence. 

Technical  aptitude,  the  independent  variable,  was 
a  composite  interval  level  variable.  For  the  purpose  of 
this  study,  the  construct  of  technical  aptitude  included 
the  following  two  characteristics: 

1.  Program  manager  inquisitiveness  on  technical 
program  aspects. 

2.  Program  manager  quickness  in  assimilating  new 
technical  information. 


Both  the  program  manager  and  technical  advisor  survey 
instruments  included  two  items  used  to  measure  perceived 
technical  aptitude  (one  pertaining  to  each  of  the  above 
characteristics) .  Responses  to  each  item  were  rated  on  a 
five-point  scale.  The  sum  of  the  individual  ratings  was 
taken  as  the  value  of  the  composite  variable  termed  a  tech¬ 
nical  aptitude  index. 

The  dependent  variable  was  perceived  program  mana¬ 
ger  technical  competence  which  was  measured  as  described 
under  Hypothesis  1  above. 

Statistical  Testing.  Correlation  analysis 
was  used  to  calculate  a  correlation  coefficient  (Pearson  r) 
for  the  program  manager  self-ratings  of  technical  aptitude 
and  technical  competence.  Correlation  analysis  of  the  tech 
nical  advisor  ratings  of  program  manager  technical  aptitude 
and  technical  competence  were  intended  as  well.  The  null 
hypothesis  was  that  the  correlation  coefficients  would  not 
be  significantly  greater  than  zero. 


Hypothesis  3 .  This  hypothesis  was  used  to  test 
the  relationship  between  acquisition  phase  of  the  program 
being  managed  and  the  importance  of  the  program  manager 
having  a  high  level  of  technical  competence.  The  null 
hypothesis  was  that  ratings  of  the  importance  of  program 
managers  having  a  high  level  of  technical  competence  would 


not  be  significantly  different  for  programs  in  different 
phases  of  the  acquisition  cycle. 

Program  phase,  the  independent  variable,  was 
treated  as  a  nominal  variable.  Programs  were  placed  in 
one  of  five  categories  based  on  the  acquisition  phase  pro¬ 
gram  managers  indicated  their  programs  were  in  at  the  time 
of  data  collection.  The  five  categories  were: 

1.  Category  A — Concept  Definition 

2.  Category  B — Demonstration  and  Validation 

3.  Category  C--Full-scale  Development 

4 .  Category  D — Production  and  Deployment 

5.  Category  E — Operations  Support 

The  dependent  variable,  importance  of  the  program 
manager  having  a  high  level  of  technical  competence,  was  a 
composite  variable  treated  at  the  interval  level.  Two 
items  were  included  in  both  the  program  manager  and  tech¬ 
nical  advisor  survey  instruments  for  measuring  this  vari¬ 
able.  One  item  asked  for  a  straightforward  rating  of 
the  importance  of  the  program  manager  having  a  high  level 
of  technical  competence  relative  to  other  factors.  The 
other  item  questioned  whether  a  lack  of  technical  compe¬ 
tence  on  the  part  of  the  program  manager  was  likely  to  be 
a  serious  detriment  to  the  program.  Responses  to  each  item 
were  rated  on  a  five-point  scale.  The  sum  of  the  two 
ratings  was  taken  as  the  value  of  the  composite  variable 


which  was  termed  a  program  manager  technical  competence 
importance  index. 


Statistical  Testing.  Unbalanced  single¬ 
factor  analysis  of  variance  (ANOVA)  at  a  .05  level  alpha 
was  used  to  determine  if  a  significant  difference  existed 
in  values  of  the  dependent  variable  between  two  or  more 
educational  categories.  The  ANOVA  F-test  indicates  whether 
a  significant  difference  exists  between  at  least  two  cate¬ 
gories,  but  does  not  identify  where  the  differences  occur. 
The  Bonferroni  multicomparison  technique  was  then  used  to 
identify  between  which  categories  differences  existed. 
Bonferroni  was  used  because  the  problem  was  unbalanced. 
Since  the  Bonferroni  technique  controls  Type  I  error  well, 
but  does  not  control  Type  II  errors  as  well  as  other  tech¬ 
niques,  meaning  some  significant  differences  could  be 
masked,  a  fairly  large  (e.g.,  .1)  level  alpha  was  used. 

The  testing  was  intended  to  be  done  twice;  once 
using  the  program  manager  self-ratings  and  once  using  the 
technical  advisor  ratings  of  importance  of  program  manager 
technical  competence. 

Hypothesis  4 .  This  hypothesis  was  used  to  test 
whether  more  than  50  percent  of  program  managers  and  tech¬ 
nical  advisors  believed  technical  competence  should  be  a 
consideration  in  selecting  officers  for  high-level  program 
management  positions.  The  null  hypothesis  was  that 


significantly  more  than  50  percent  of  program  managers  and 
their  technical  advisors  do  not  believe  it  should  be  a  con¬ 


sideration.  Each  of  the  program  manager  and  technical 
advisor  survey  instruments  included  an  item  which  described 
the  AFSCR  36-5  procedure  for  selecting  officers  for  the 
Acquisition  Managers  List  (AML)  and  the  Senior  Acquisition 
Managers  List  (SAML)  to  assess  whether  or  not  each  inter¬ 
viewee  thought  technical  competence  should  also  be  a  con¬ 
sideration  in  this  selection  process. 

Statistical  Testing.  A  binomial  test  of 
proportions  was  done  to  determine  whether  the  proportion  of 
"yes"  responses  was  significantly  greater  than  .5  at  a  .1 
level  alpha.  The  test  was  conducted  separately  for  program 
manager  responses  and  for  technical  advisor  responses. 

Reliability  and  Validity 

Several  actions  were  taken  to  promote  and  assess 
the  reliability  and  validity  of  this  study. 

Sample  Size.  Genera  1 izabil ity  of  sample  statistics 
to  the  population  is  largely  a  function  of  sample  and  popu¬ 
lation  sizes.  A  goal  of  55  program  manager  interviews  was 
set  so  that  a  90  percent  ±  10  percent  confidence  level  could 
be  achieved  for  this  study  as  calculated  for  conducting  a 
test  of  proportions  as  in  testing  on  Hypothesis  4.  This 


sample  size  was  based  on  a  finite  population  of  275  total 
AFSC  2996  positions  (4:11-14). 


Measurement  Reliability.  Each  of  the  variables 
which  involved  ratings  of  opinions/perceptions  (e.g., 
program  manager  self-ratings  of  technical  competence)  was 
a  composite  variable  based  on  at  least  two  survey 
instrument  items.  Cronbach  alpha  reliability  coefficients 
were  computed  for  each  of  these  composite  variables  as  an 
indication  of  the  measurement  reliability  of  the  instru¬ 
ments. 


Convergent  Validity.  Convergent  validity  is  a 
measure  of  how  well  separate  instruments  measure  the  same 
construct  or  of  how  similarly  separate  groups  view  the 
same  phenomenon.  For  this  research  effort,  the  program 
manager  and  technical  advisor  survey  instruments  contained 
the  same  items  (with  minor  word  changes)  for  measuring 
opinion/perception  variables.  Pearson  correlation  coeffi¬ 
cients  were  computed  for  program  manager  versus  technical 
advisor  ratings  of  program  manager  technical  competence, 
program  manager  technical  aptitude,  and  perceived  importance 
of  the  program  manager  having  a  high  level  of  technical  com¬ 
petence.  The  strength  of  these  correlations  provided  an 
indication  of  how  accurately  these  constructs  were  measured. 
In  addition,  paired  t-tests  were  conducted  to  determine  if 


technical  advisor  ratings  were  significantly  different  than 
program  manager  ratings  in  any  of  these  three  rated  areas. 

Summary 

A  convenience  sampling  technique  was  employed  in 
collecting  data  from  program  managers  (those  in  AFSC  2996 
positions)  and  their  technical  advisors.  The  method  of 
data  collection  was  through  personal  interviews.  Sta¬ 
tistical  analyses  were  performed  in  testing  four  hypotheses 
which  related  directly  to  the  research  questions  that  had 
been  posed.  Additional  statistical  analyses  were  used  to 
indicate  the  level  of  measurement  reliability  of  the  instru' 
ments  used  and  the  degree  of  convergent  validity  on  key 


constructs. 


IV.  Results 


Introduction 

A  total  of  32  interviews  were  conducted;  26  with 
program  managers  and  six  with  technical  advisors.  Of  the 
70  AFSC  2996  positions  identified  within  Aeronautical  Sys¬ 
tems  Division  (ASD) ,  13  were  vacant  and  four  had  security 
restrictions  which  prohibited  scheduling  of  interviews. 
Interviews  were  not  conducted  with  the  remaining  27  pro¬ 
gram  managers  due  to  time  constraints  and  scheduling  prob¬ 
lems.  Technical  advisor  interviews  were  particularly  dif¬ 
ficult  to  arrange.  Some  statistical  tests  proposed  in 
Chapter  III  using  data  collected  from  technical  advisors 
were  not  warranted  due  to  the  low  number  of  technical 
advisor  responses.  Those  technical  advisor  interviews  that 
were  conducted  were  used  to  check  for  convergent  validity 
with  respective  program  manager  responses. 

The  program  managers  interviewed  were  clearly  sup¬ 
portive  and  interested  in  the  topic  of  this  study.  Inter¬ 
views  which  could  have  been  completed  within  twenty  minutes 
often  lasted  over  an  hour  at  the  interviewee's  preference. 
Much  in  the  way  of  qualitative  perceptions  and  opinions 
was  recorded  in  addition  to  the  formal  interview  responses 
(14:131-153) . 
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Hypothesis  1.  Hypothesis  1  stated  that  the  ratings 
of  program  manager  technical  competence  would  differ  sig¬ 
nificantly  depending  upon  academic  educational  background. 

Statistical  Results.  This  hypothesis  was 
supported  by  statistical  analysis.  The  analysis  of  vari¬ 
ance  (ANOVA)  F  test  resulted  in  a  p-value  of  .0015  which 
was  much  smaller  than  the  .01  level  alpha  used  in  making 
the  test.  This  indicated  a  statistically  significant  dif¬ 
ference  in  ratings  of  program  manager  perceived  technical 
competence  between  at  least  two  of  the  academic  degree 
categories . 

Table  1  shows  Bonferroni  multicomparison  results 
conducted  at  a  .1  level  alpha.  This  indicates  that  sig¬ 
nificant  differences  in  technical  competence  ratings 
existed  between  those  in  category  A  (electrical,  computer, 
and  aeronautical  engineering)  and  category  C  (business/ 
management  and  other  B.S./M.S.  degrees  not  in  categories 
A  or  B)  and  between  category  A  and  category  D  (all  other 
nontechnical  degrees) .  No  significant  difference  was  indi¬ 
cated  between  category  B  (other  engineering,  physical 
sciences,  math,  and  computer  science)  and  any  of  the  other 
categories  nor  between  categories  C  and  D. 

Qualitative  Findings.  Ten  of  the  twenty- 
six  program  managers  interviewed  (38  percent)  did  not  meet 


TABLE  1 


BONFERRONI  MULTICOMPARISON  OF  PROGRAM 
MANAGER  TECHNICAL  COMPETENCE 


Academic 

Degree 

Category 

Comparison 

Simultaneous 

Lower 

Confidence 

Limit 

Difference 

Between 

Means 

Simultaneous 

Upper 

Conf idence 
Limit 

A  -  B 

-3.263 

2.250 

7.763 

> 

1 

D 

0.253 

6.417 

12.581*** 

A  -  C 

3 . 066 

7.607 

12.149*** 

B  -  A 

-7.763 

-2.250 

3.263 

B  -  D 

-3.127 

4.167 

11.460 

B  -  C 

-0.628 

5.357 

11.342 

D  -  A 

-12.581 

-6.417 

-0.253*** 

D  -  B 

-11.460 

-4.167 

3.127 

D  -  C 

-5.399 

1.190 

7.780 

C  -  A 

-12.149 

-7.607 

-3.066*** 

C  -  B 

-11.342 

-5.357 

0.628 

C  -  D 

-7.780 

-1.190 

5.399 

Alpha  =  0.1 

Confidence  =  0 

.9  DF  =  22 

MSE  =  13.5806 

Critical  Value 

of  t  =  2.59121 

NOTE: 

***Comparisons 

significant  at 

the  .01  level. 
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mandatory  AFR  36-1  education  prerequisites  for  entry  into 
the  AFSC  2996  specialty  (7) .  Figure  1  shows  a  plot  of  pro¬ 
gram  manager  technical  competence  ratings  for  each  of  the 
academic  educational  degree  categories.  Twelve  of  the 
twenty-six  program  managers  interviewed  had  electrical  or 
aeronautical  engineering  degrees.  None  of  the  officers 
interviewed  had  computer  engineering  or  computer  science 
degrees.  All  of  the  officers  in  category  C  had  at  least 
one  business  or  management  degree.  Again,  since  this  was 
ex  post  facto  research,  these  results  do  not  necessarily 
mean  that  engineering  degrees  cause  high  technical  compe¬ 
tence.  Indeed,  some  of  those  in  categories  A  and  B  did  not 
rate  particularly  high  in  technical  competence  while  one 
category  D  officer  rated  very  high. 

Hypothesis  2.  Hypothesis  2  stated  that  ratings  of 
program  manager  technical  competence  would  be  signifi¬ 
cantly  correlated,  in  a  positive  sense,  with  ratings  of 
program  manager  technical  aptitude. 

Statistical  Results.  This  hypothesis  was 
strongly  supported  by  the  statistical  analysis.  A  Pearson  r 
correlation  coefficient  of  .86  was  obtained  indicating  a 
very  strong  correlation  between  perceived  intrinsic  tech¬ 
nical  aptitude  and  perceived  technical  competence. 
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A  B  c  D 

Academic  Degree  Category 

Legend 

A — Electrical,  computer,  or  aerospace  engineering. 
B — Other  engineering,  physical  sciences,  math  or 
computer  science. 

C — Business/management  and  other  B.S./M.S.  degrees 
D — All  other  nontechnical  (e.g.,  B.A.)  degrees. 

Fig.  1.  Program  Manager  Technical  Competence  versus 
Academic  Degree  Category 
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Qualitative  Findings.  Figure  2  shows  a 
plot  of  program  manager  technical  competence  indices  versus 
technical  aptitude  indices.  Data  points  are  coded  A,  B,  C, 
and  D  according  to  academic  education  background.  This 
relationship  may  serve  as  an  example  as  to  why  it  is  not 
proper  to  assume  from  this  study  that  an  engineering  educa¬ 
tion  causes  high  technical  competence.  Perhaps  those 
with  high  technical  aptitude  tend  to  pursue  technical  educa¬ 
tions.  The  category  D  officer  with  a  high  technical  compe¬ 
tence  rating  (see  Figure  1) ,  while  possessing  only  an 
undergraduate  degree  in  Public  Relations,  had  a  very  high 
rating  of  intrinsic  technical  aptitude  (note  the  data 
point  marked  with  an  in  Figure  2) .  Again,  this  research 

was  not  structured  as  an  experiment  and  cannot  show 
causality . 

Hypothesis  3 .  Hypothesis  3  stated  that  ratings  of 
the  perceived  importance  of  a  program  manager  having  a 
high  level  of  technical  competence  would  differ  signifi¬ 
cantly  depending  upon  the  program  phase  of  the  system  being 
acquired. 


Statistical  Results.  This  hypothesis  was 
not  supported  by  statistical  analysis.  The  ANOVA  F  test 
resulted  in  a  p-value  of  .61.  This  was  well  above  the  .05 
level  alpha  established  in  defining  the  test.  Further 
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Technical  Aptitude  Index 

Legend 

A — Electrical,  computer,  or  aerospace  engineering. 
B — Other  engineering,  physical  sciences,  math  or 
computer  science. 

C — Business/management  and  other  B.S./M.S.  degrees 
not  included  in  A  or  B  above. 

D — All  other  nontechnical  (e.g.,  B.A.)  degrees- 


Fig.  2.  Program  Manager  Technical  Competence  versus 

Technical  Aptitude 


analysis  using  the  Bonferroni  multicomparison  technique 
was  not  warranted. 

Qualitative  Findings.  Research  indicating 
increased  levels  of  management  concern  in  dealing  with 
technical  issues  during  system  design  and  development  was 
not  quantitatively  substantiated  by  this  study  (20) .  Most 
of  the  program  managers  interviewed  were  involved  primarily 
with  programs  in  the  full-scale  development  phase.  Two 
program  managers  focused  on  programs  in  the  concept  defini¬ 
tion  phase,  three  on  the  demonstration/ validation  phase, 
three  on  the  production  and  deployment  phase,  and  one  on 
the  operations  support  phase.  Given  the  small  number  of 
observations  in  each  of  these  categories,  this  study 
certainly  does  not  contradict  other  research  in  this 
area  (20)  . 

Many  of  the  program  managers  interviewed  indicated 
that  often  the  technical  issues  receiving  high-level  man¬ 
agement  attention  arose  during  system  test,  the  start  of 
production,  or  upon  initial  operational  use.  However,  the 
consensus  was  that  most  truly  important,  although  often 
obscure,  technical  considerations  having  long-term  signifi¬ 
cance  were  addressed  during  full-scale  development. 

Hypothesis  4 .  Hypothesis  4  stated  that  a  majority 
of  those  in  program  manager  positions  and  their  technical 
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advisors  would  feel  that  technical  competence  should  be 
part  of  the  criteria  used  to  select  officers  for  key  middle 
management  and  senior  program  management  positions. 


Statistical  Results.  This  hypothesis  was 
statistically  supported  for  program  manager  responses. 
Seventeen  of  the  26  program  managers  interviewed  (65  per¬ 
cent)  indicated  that  they  felt  technical  competence  should 
be  a  consideration  in  the  AFSCR  36-5  selection  process. 

The  binomial  test  of  proportions  resulted  in  a  p-value  of 
.04  which  was  less  than  the  .1  level  alpha  of  the  test. 

Due  to  the  small  number  of  technical  advisor  interviews, 
this  hypothesis  was  not  tested  with  technical  advisor 
response  data. 


Qualitative  Findings.  The  question  associ¬ 
ated  with  this  hypothesis  was  purposefully  intended  to 
solicit  prograun  managers'  opinions  of  the  AFSCR  36-5 
Acquisition  Management  Career  Development  Program  (AMCDP) . 
The  view  which  seemed  to  be  held  by  most  of  those  inter¬ 
viewed  was  that  the  certif ication  process  of  the  AMCDP 
would  probably  have  a  very  positive  future  effect  by 
encouraging  officers  in  the  acquisition  business  to  seek 
more  schooling  and  a  broader  range  of  assignments  than 
might  otherwise  be  the  case.  The  selection  process  was 
viewed  somewhat  less  optimistically.  A  general  displeasure 
with  selection  based  on  "square-filling"  was  expressed. 


A  recurring  theme  of  the  discussions  was  that  an  indi¬ 
vidual's  specific,  unique  set  of  experiences  and  capabili¬ 
ties  should  be  considered  as  a  basis  for  selection  to  either 


of  the  AMCDP  lists  and  especially  for  job  assignments. 
Program  managers  seemed  to  feel  strongly  that  matching 
individual  officer  qualifications  and  strengths  with  par¬ 
ticular  job  needs  had  a  high  payoff. 

A  typical  response  to  the  question  associated  with 
Hypothesis  4  was,  "Yes,  technical  competence  should 
definitely  be  a  consideration,  but  how  would  it  be  mea¬ 
sured?"  Proper  evaluation  and  documentation  of  technical 
competence  (i.e.,  measuring  the  aspects  truly  important 
for  program  management)  appeared  to  be  a  much  warranted 
concern.  Those  interviewed  invariably  noted  that  added 
"square-f illing"  as  a  means  of  certifying  technical  compe¬ 
tence  would  be  highly  undesirable.  Many  of  those  respond¬ 
ing  negatively  to  the  Hypothesis  4  question  said  that  tech¬ 
nical  competence  was  important,  but  that  it  would  hopefully 
be  factored  into  the  "duty  performance"  criterion. 

Other  Findings 

Importance  of  SPO  Director  Technical  Competence. 

In  the  very  early  phases  of  data  collection  for  this  study 
it  became  apparent  that  those  officers  who  functioned  as 
SPO  directors  placed  a  much  higher  priority  on  the  need 
for  technical  competence  in  doing  their  jobs  than  did 


program  managers  who  provided  higher  level  administrative 
supervision  of  multiple  programs.  The  SPO  directors  often 
mentioned  how  technical  abilities,  in  addition  to  business 
and  interpersonal  skills,  were  needed  to  interact  effec¬ 
tively  with  the  contractor  and  their  own  engineers,  to  make 
appropriate  decisions  on  trading  off  cost,  schedule,  per¬ 
formance,  and  risk,  and  to  maintain  credibility  with  upper 
organizational  levels.  Those  in  AFSC  2996  positions  who 
were  not  SPO  directors,  however,  usually  described  their 
jobs  as  strictly  business  and  administration  with  an 
emphasis  on  finance  but  generally  having  little  to  do  with 
technical  issues. 

Figure  3  shows  ratings  of  the  importance  of  tech¬ 
nical  competence  for  SPO  directors  and  for  non-SPO  director 
program  managers.  A  t-test  comparing  the  technical  compe¬ 
tence  importance  indices  of  these  groups  resulted  in  a 
p-value  of  .01  indicating  that  the  difference  is  indeed 
statistically  significant. 

An  explanation  of  what  the  technical  competence 
importance  index  scale  represents  may  be  in  order.  This 
index  was  based  on  program  manager  responses  to  two  instru¬ 
ment  questions.  Asked  to  assume  they  were  involved  in 
choosing  a  successor  for  their  jobs,  the  first  question 
asked  how  important  a  consideration  technical  competence 
would  be  in  that  selection.  The  second  question  asked  how 
serious  a  detriment  it  might  be  if  the  chosen  successor 
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lacked  technical  ability  but  was  highly  qualified  in  terms 
of  operational  experience  and  management  expertise.  A 
rating  of  "10"  on  the  Figure  3  index  indicates  that  the 
program  manager  responses  were  that  technical  competence 
would  be  the  "most  important  consideration"  and  that  a  lack 
of  technical  competence  "would  definitely  seriously  hinder 


the  program."  An  index  of  "2"  would  indicate  that  tech¬ 
nical  competence  was  "not  a  factor  at  all"  and  that  a  lack 
of  technical  competence  "would  not  make  a  difference  at 
all."  An  index  of  "6"  represents  a  neutral  position.  A 
response  that  consideration  of  technical  competence  would  be 
"even  with  other  factors"  and  that  a  lack  of  technical 
competence  "would  be  a  detriment,  but  not  serious"  would 
be  one  way  of  arriving  at  the  neutral  index. 

Technical  Areas  for  Training/Education  Emphasis. 

In  addition  to  the  formal  interview  questions  used  for  sta¬ 
tistical  testing  and  the  spontaneous  opinions  expressed  and 
recorded  during  the  interviews,  one  completely  open-ended 
question  was  asked  in  each  interview  (14:131-153).  The 
purpose  of  this  question  was  to  identify  technical  areas 
warranting  increased  emphasis  in  the  training  and  education 
of  SPO  director  candidates  (11) .  The  number  of  areas  men¬ 
tioned  by  a  single  program  manager  ranged  from  none  to  five 
with  most  officers  naming  two  areas  needing  more  emphasis. 


As  Table  2  shows,  areas  associated  with  computers. 


electronics,  and  basic  science  and  engineering  were  the 
most  frequently  mentioned.  While  at  first  glance  it  may 
seem  surprising  that  areas  like  reliability,  maintainabil¬ 
ity,  and  producibility  were  mentioned  less  often,  it  should 
be  noted  that  much  has  been  done  recently  to  promote  the 
appreciation  of  the  importance  of  these  disciplines  while 
this  question  specifically  asked  for  areas  warranting 
increased  emphasis. 

Reliability  and  Convergent  Validity 

Sample  Size.  While  the  actual  sample  size  was 
smaller  than  the  goal  that  had  been  set  for  this  study, 
it  was  large  enough  to  obtain  slightly  better  than  an  85 
percent  i  15  percent  confidence  level  as  calculated  for 
the  Hypothesis  4  test  of  proportions  (4:11-14). 

Measurement  Reliability.  Cronbach  alpha  relia¬ 
bility  coefficients  were  calculated  for  the  program  manager 
survey  instrument.  The  reliability  coefficients  for  the 
five-item  technical  competence  index  was  .94.  A  .82 
Cronbach  alpha  was  calculated  for  the  two-item  technical 
aptitude  index.  The  two-item  technical  competence  impor¬ 
tance  index  had  a  .87  Cronbach  alpha. 

Convergent  Validity.  For  the  cases  where  technical 
advisor  interviews  were  conducted,  Pearson  r  correlation 


TABLE  2 


TECHNICAL  AREAS  NEEDING  EMPHASIS 


Responses  to  the  question:  "What  specific  technical  areas 
do  you  believe  may  warrant  increased  emphasis  in  the  train¬ 
ing  and  education  of  officers  who  will  become  SPO  directors 
in  the  future?" 


Area  Times  Mentioned 

Computer  Software  Development  .  14 

Basic  Electronics  .  7 

General  Engineering  or  Science  .  6 

Computer  Hardware  Technology  .  5 

Systems  Engineering  (Integration)  .  4 

Aerodynamics  .  3 

Test  and  Integration .  3 

Reliability  and  Maintainability  .  2 

Producibility  .  2 

Avionics  .  2 

Radar  Theory .  1 

Materials  .  1 

Engine  Technology  .  1 

Electronic  Combat  .  1 

Operations  Analysis  .  1 

Inertial  Navigation  Theory  .  1 
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coefficients  were  calculated  to  assess  the  level  of  program 
manager  and  technical  advisor  convergent  validity  associated 
with  the  three  indices.  The  resulting  Pearson  r  values  were 
.94  for  perceived  technical  competence,  .92  for  perceived 
technical  aptitude,  and  .97  for  perceived  importance  of 
program  manager  technical  competence. 


V.  Findings  and  Recommendations 


Assumptions  and  Limitations 

The  primary  assumption  that  must  be  made  in  order 
to  generalize  the  findings  of  this  study  to  program  manager 
positions  outside  of  Aeronautical  Systems  Division  (ASD) 
is  that  the  sample  used  was  representative  of  the  popula¬ 
tion  as  a  whole.  While  the  sample  size  was  smaller  than 
desired  (about  10  percent  of  the  population)  and  was 
limited  to  ASD  officers,  many  of  those  interviewed  had 
worked  in  other  Systems  Command  product  divisions  and  did 
not  indicate  that  ASD  program  management  was  unique  with 
respect  to  the  questions  asked.  With  regard  to  the  ques¬ 
tion  of  technical  areas  warranting  increased  emphasis  for 
SPO  director  candidates,  however,  one  suspects  that  program 
managers  from  other  product  divisions  (Space  Division,  for 
example)  might  provide  some  additional  responses. 

An  important  limitation  to  recognize  is  that  since 
this  was  ex  post  facto  research  and  not  an  experiment,  this 
study  does  not  show  causality.  In  particular,  it  cannot 
be  said  that  certain  types  of  academic  education  cause 
higher  levels  of  technical  competence  in  program  managers 
based  upon  this  research. 


Key  Findings 

The  key  findings  resulting  from  this  research  were 

1-  Air  Force  acquisition  program  managers  with 
Electrical  Engineering  and  Aerospace  Engineering  academic 
degrees  rate  significantly  higher  in  perceived  technical 
competence  than  do  those  with  liberal  arts  and  business- 
related  degrees. 

2.  There  is  a  strong  correlation  between  the  per¬ 
ceived  intrinsic  technical  aptitude  of  program  managers 
and  their  perceived  technical  competence. 

3.  A  significant  number  of  officers  (38  percent 
of  those  sampled)  in  AFSC  2996  positions  do  not  meet 
mandatory  educational  prerequisites  for  entry  into  the 
Program  Manager  specialty  per  AFR  36-1. 

4.  The  importance  of  having  a  high  level  of  tech¬ 
nical  competence  is  greater  for  SPO  directors  (those 
functioning  as  the  single  manager  for  developing  and 
delivering  a  system)  than  for  program  managers  who  provide 
higher  level  administrative  supervision  of  multiple  pro¬ 
grams  . 

5.  Technical  competence  should  be  a  consideration 


in  the  Acquisition  Management  Career  Development  Program 
(AMCDP)  selection  process,  but  more  investigation  may  be 
required  to  identify  a  proper  method  for  doing  so. 


6.  Increased  emphasis  may  be  warranted  in  educa¬ 
tion  or  training  SPO  director  candidates  in  the  areas  of 
computer  hardware  and  software,  basic  electronics,  and 
general  science  and  engineering  principles.  The  develop¬ 
ment  and  integration  of  computer  software  is  an  area  war¬ 
ranting  particular  emphasis. 

Recommendations 

The  following  areas  are  recommended  for  considera¬ 
tion  and/or  action: 

1.  Recommend  the  Air  Force  consider  defining  two 
specializations  within  the  29XX  Program  Management  utiliza¬ 
tion  field;  one  for  officers  who  function  as  SPO  directors 
for  a  single  major  acquisition  program  and  another  for  those 
with  supervisory,  more  administrative  program  management 
responsibilities.  This  change  could  serve  to  delineate  the 
different  roles  and  responsibilities  of  these  jobs  and 
might  help  to  clarify  reporting  relationships  to  the  new 
Program  Executive  Officer  (PEO)  positions. 

2.  Recommend  the  Air  Force  resolve  the  discrepancy 
between  the  AFR  36-1  educational  requirements  for  the  AFSC 
2996  specialty  and  the  real-world  situation.  A  review  of 
the  educational  requirements  could  be  accomplished  in  con¬ 
junction  with  the  abov^e  recommendation  allowing  a  relaxa¬ 
tion  of  the  technical  focus  for  the  non-SPO  director 


specialty.  Perhaps  a  formal  waiver  process  would  be  in 
order  in  the  near-term. 


3.  Recommend  those  involved  in  administering  the 
AMCDP  selection  process  and,  perhaps  more  importantly, 
those  involved  in  assigning  selected  officers  to  key  pro¬ 
gram  management  positions  make  a  serious  effort  to  appropri¬ 
ately  consider  technical  factors  when  matching  individual 
officers  with  jobs. 

Suggestions  for  Further  Research 

The  following  areas  are  suggested  for  further 
research; 

1.  The  method  used  in  this  research  effort  could  be 
applied  to  a  different,  hopefully  larger,  sample  of  program 
managers.  One  variation  might  be  to  attempt  to  identify 
differences  in  technical  competence  requirements  for  SPO 
directors  at  different  Systems  Command  product  divisions. 
While  engineering  support  at  ASD  is  provided  by  military 
and  civil  service  personnel.  Space  Division  and  Electronic 
Systems  Division  contract  for  technical  support  with 
Federal  Contract  Research  Corporations  (i.e.,  the  Aerospace 
Corporation  and  MITRE  Corporation) .  Perhaps  this  differ¬ 
ence  affects  the  level  to  which  SPO  directors  feel  the  need 
to  personally  address  technical  program  issues.  Another 
variation  might  be  to  investigate  other  moderating  factors, 
such  as  the  degree  to  which  a  program  pushes  the  state  of 


the  art,  that  might  be  associated  with  the  need  for  a  high 
level  of  SPO  director  technical  competence. 

2.  Evaluation  of  program  manager  technical  compe¬ 
tence  for  selection  and  assignment  purposes  is  an  unresolved 
issue.  An  examination  of  how  this  might  be  accomplished 
practically  is  in  order.  A  related  concern  is  how  to  keep 
SPO  directors  current  in  important  technical  areas  given 
the  rapid  pace  of  technological  change  without  an  undue 
overemphasis  of  things  technical.  Research  could  be  con¬ 
ducted  to  identify  the  suggested  content  and  administration 
of  a  program  manager  technical  training  effort. 

3.  The  value  and  purpose  of  operational  experi¬ 
ence  for  acquisition  program  managers  was  frequently  men¬ 
tioned  in  the  interviews  conducted  for  this  study.  Many 
disparate  views  were  expressed.  There  was  some  concern  that 
operational  tours  might  be  treated  as  AMCDP  "squares"  with¬ 
out  sufficient  regard  for  the  real  benefit  derived  from  a 
particular  operational  assignment  to  a  program  manager. 

For  example,  the  value  of  missile  experience  to  a  fighter 
aircraft  program  manager  was  questioned.  An  exploratory 
study  could  be  conducted  to  address  the  importance  of 
operational  experience  for  acquisition  program  managers  and 
the  associated  moderating  factors. 


Conclusion 


An  Air  Force  officer  needs  a  diversity  of  knowledge 
and  a  variety  of  talents  to  excel  as  an  acquisition  program 
manager  (2;  7;  12).  This  study  explored  one  attribute  to 
be  considered  in  prospective  program  managers;  that  of 
technical  competence.  Extremely  capable,  versatile,  multi¬ 
talented  officers  will  be  needed  to  fill  SPO  director 
positions  in  an  age  of  rapidly  changing  technology,  more 
direct  reporting  channels,  and  increased  personal  authority 
(16:61,73).  Technical  competence  may  not  be  the  single 
most  important  quality  in  future  SPO  directors,  but,  in  an 
effort  to  groom  and  select  the  best  of  the  best  to  manage 
major  system  acquisition  programs,  technical  ability  must 
certainly  be  a  serious  consideration. 


1 .  Educational  Degree (s) 

What  type  of  undergraduate  degree  do  you  have? 

Any  particular  area  of  specialization? 

What  post-graduate  degrees? 

2 .  Program  Phase 

What  phase  is  your  program  currently  in  (i.e..  Con¬ 
cept  Definition,  Demonstration /Validation,  Full-scale 
Development,  Production  and  Deployment,  or  Operations 
Support) ? 

3 .  Self-rated  Technical  Competence 

In  the  discussion  that  follows,  the  term  "technical 
competence"  will  be  used.  The  meaning  of  that  term 
is  the  ability  to  assimilate  and  use  technical  infor¬ 
mation.  Here,  it  applies  specifically  to  your  ability 
as  a  Program  Manager  to  understand  technical  concepts 
at  a  level  of  detail  commensurate  with  your  management 
position  and  to  properly  factor  that  knowledge  into 
your  decision  making.  The  technical  competence  you 
have  may  be  due  to  a  variety  of  factors  such  as  edu¬ 
cation,  experience,  or  your  own  intrinsic  aptitude. 
Please  keep  in  mind  that  the  following  questions  do 
not  deal  with  how  well  you  do  your  job  as  a  Program 
Manager.  The  focus  is  on  how  comfortable  you  are  with 
your  own  ability  to  deal  with  technical  issues. 

3.1  How  well  do  you  meet  your  own  personal  expectations 
for  technical  expertise  in  doing  this  job? 

3.2  Imagine  that  program  managers  had  apprentices.  To 
what  degree  would  the  technical  competence  you 
exhibit  be  a  good  model  for  your  apprentice? 

3.3  To  what  extent  are  your  talents,  or  the  places  you  can 
best  concentrate  your  attention,  in  areas  other  than 
technical  aspects  of  the  program? 

3.4  Considering  the  time  you  have  spent  in  your  position, 
how  familiar  are  you  with  the  key  technical  aspects 
of  the  pr  Dgram? 


3.5  Do  you  feel  you  have  all  the  technical  skills  you 
need  to  perform  well  in  your  job? 


4. 


Self-rated  Technical  Aptitude 


4.1  How  strong  is  your  natural  tendency  to  ask  questions 
or  solicit  information  concerning  technical  aspects 
of  your  program? 

4.2  How  quickly  would  you  say  you  "come  up  to  speed"  as 
far  as  understanding  technical  information  that  is 
new  to  you? 

5 .  Importance  of  Technical  Competence 

Assume  you  got  orders  to  leave  this  job  within  sixty 
days  and  were  involved  in  choosing  your  replacement 

5.1  How  important  a  consideration  do  you  think  it  should 
be  that  your  replacement  have  a  high  level  of  tech¬ 
nical  competence? 

5.2  Suppose  the  person  chosen  to  replace  you  had  a  great 
deal  of  operational  experience  and  management  exper¬ 
tise  but  had  trouble  grasping  technical  issues.  Do 
you  think  his  lack  of  technical  ability  might  be  a 
serious  detriment  to  your  program? 

6 .  Technical  Competence  as  a  Selection  Criterion 

The  new  Air  Force  Systems  Command  Acquisition  Manage¬ 
ment  Career  Development  Program  has  provisions  for 
selecting  officers  to  assume  senior  program  management 
positions  on  a  "best  qualified"  basis.  There  is  to 
be  an  Acquisition  Managers  List  of  100  officers  and 
a  Senior  Acquisition  Managers  List  of  50  officers 
selected  each  year.  Duty  performance,  leadership 
ability,  and  operational  experience  are  to  be  weighted 
heavily  in  this  selection  process.  Do  you  think  tech¬ 
nical  competence  should  be  considered  as  well? 

Technical  Training/Education  Suggestions 
What  specific  technical  areas  do  you  believe  may 
warrant  increased  emphasis  in  the  training  and  educa¬ 
tion  of  officers  who  will  become  SPO  directors  in  the 
future? 


7. 


Program  Manager  Name:  _ 
Technical  Advisor  Name: 


Rank : 


Organization/Program:  _  Category: _ 

Category  Definitions:  M — Multiple  program  oversight 

S — Single  program  responsi- 
bil ity 

1.  Educational  Degree (s)  Category: _ 

Undergraduate:  _ 

Graduate:  _ 

Category  Definitions:  A--Electr ical ,  computer,  or 

aerospace  engineering. 

B--Other  engineering,  physical 
sciences,  math  or  computer 
science . 

C — Business/management  and 

ether  B.S./M.S.  '^egrees  not 
included  in  A  or  B  above . 

D--A11  other  nontechnical 
(e.g.,  B.A.)  degrees* 


Program  Phase 


Category : 


Category  Definitions:  A — Concept  Definition 

B — Demonstration/ Val idat ion 
C — Full-scale  Development 
D — Production  and  Deployment 
E — Operations  Support 


Self-rated  Technical  Competence 


3  •  1  Personal  Expectations 

Rating  scale:  1 — Not  very  well 

2 —  Marginally 

3 —  Adequately 

4— -Fairly  well 

5 —  Extremely  well 


Rating : 
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Rating : 


3 . 2  Model  for  Apprentice 
Rating  Scale:  1 — Poor  model 

2 —  Fair  model 

3— -Acceptable  model 

4 —  Pretty  good  model 

5 —  Exceptional  model 

3.3  Talents  Elsewhere  Rating: _ 

Rating  Scale:  1 — Far  more  talented  in  other  areas 

2 —  More  talented  in  other  areas 

3 —  Balanced 

4— -Technical  leaning 

5 —  Most  talented  in  technical  aspects 

3.4  Familiarity  Considering  Time  Rating: _ 

Rating  Scale:  l--Not  familiar  at  all 

2- -Somewhat  familiar 

3- -Familiar  with  key  issues 

4 -  Quite  familiar  with  key  issues 

5- -Intimately/ thoroughly  familiar 

with  all  technical  issues 

3.5  Technical  Skills  Rating: _ 

Rating  Scale:  l--Do  not  have  needed  technical 

skills  at  all 

2 —  Need  more  technical  skills  to 
perform  well 

3 —  Have  some  technical  skills  but 
more  would  be  helpful 

4- -Have  most  of  needed  technical 

skills 

5- -Have  all  of  the  technical  skills 

needed  to  perform  extremely  well 

4 .  Self-rated  Technical  Aptitude 

4.1  Propensity  to  Question  Rating; _ 

Rating  Scale:  l--Do  not  seek  technical  information 

2- -0ccasionally  inquire  about 

technical  program  aspects 

3 —  Question  technical  matters 
that  become  issues 

4 —  Tend  to  ask  questions  until 
comfortable  that  someone  understand 

5- -Strong  tendency  to  ask  questions 

until  technical  issues  are  under¬ 
stood 


4.2  Quickness  in  Learning  Rating: _ 

Rating  Scale:  1 — Do  not  understand  technical  matters 

2 —  Not  a  strong  point,  but 
eventually  understand 

3 —  Average  quickness 

4— -Above  average  quickness 

5 —  Extremely  quick  in  absorbing 
technical  information 

5 .  Importance  of  Technical  Competence 

5.1  Importance  Rating: _ 

Rating  Scale:  1 — Not  a  factor  at  all 

2 —  Not  a  very  important  factor 

3 —  Even  with  other  factors 

4 —  One  of  the  more  important  factors 

5 —  Most  important  consideration 

5.2  Detriment  Potential  Rating: _ 

Rating  Scale:  1 — Would  not  make  a  difference  at  all 

2 —  Would  be  unfortunate,  but  not 
seriously  harm  program 

3 —  Would  be  a  detriment,  but  not  serious 

4 —  Would  be  likely  to  hinder  the  program 

5 —  Would  definitely  seriously  hinder 
program 

6.  Selection  Criterion  Rating: _ 

Rating  Scale:  Y--Should  be  included 

N — Should  not  be  included 


7. 


Training/ Education 


1. 


Appendix  C:  Technical  Advisor  Survey  Instrument 


Perceived  Program  Manager  Technical  Competence 
In  the  discussion  that  follows,  the  term  "technical 
competence"  will  be  used.  The  meaning  of  that  term 
is  the  ability  to  assimilate  and  use  technical  infor¬ 
mation.  It  applies  specifically  to  a  program  manager' 
ability  to  understand  technical  concepts  at  a  level  of 
detail  commensurate  with  his/her  management  position 
and  to  properly  factor  that  knowledge  into  his/her 
decision  making.  Technical  competence  may  be  a  func¬ 
tion  of  education,  experience,  and  intrinsic  apti¬ 
tude  . 

1.1  How  well  does  the  program  manager  meet  your  expecta¬ 
tions  for  technical  competence  in  doing  his  job? 

1.2  Suppose  program  managers  had  apprentices.  Would  the 
technical  competence  exhibited  by  your  program  manager 
make  a  good  model  for  an  apprentice  to  emulate? 

1.3  To  what  extent  do  you  feel  the  program  manager's 
talents,  or  where  he  can  concentrate  his  attention 
best,  are  in  areas  other  than  technical  aspects  of 
the  program? 

1.4  Considering  the  time  spent  in  the  job,  do  you  feel  the 
program  manager  is  thoroughly  familiar  with  the  key 
technical  issues  on  the  program? 

1.5  Do  you  think  the  program  manager  has  all  the  technical 
skills  he  needs  to  perform  well  in  his  job? 

2 .  Program  Manager  Technical  Aptitude 

2.1  How  much  do  you  feel  the  program  manager  has  a  natural 
tendency  to  ask  questions  or  solicit  information  con¬ 
cerning  technical  aspects  of  the  program? 

2.2  How  quickly  does  the  program  manager  "come  up  to  speed 
as  far  as  understanding  technical  information  that  is 
new  to  him? 


3. 


3.1 


3.2 


Importance  of  Technical  Competence 

Assume  the  program  manager  got  orders  to  leave  within 
sixty  days  and  that  you  were  involved  in  choosing  his 
replacement . 


How  important  a  consideration  do  you  think  it  should 
be  that  his  replacement  have  a  high  level  of  tech¬ 
nical  competence? 


Suppose  the  person  chosen  to  replace  him  had  a  great 
deal  of  operational  experience  and  management  expertise 
but  had  trouble  grasping  technical  issues.  Do  you 
think  his  lack  of  technical  ability  might  be  a  serious 
detriment  to  the  program? 


Technical  Competence  as  a  Selection  Criterion 
The  new  Air  Force  Systems  Command  Acquisition  Manage¬ 
ment  Career  Development  Program  has  provisions  for 
selecting  officers  to  assume  senior  program  manage¬ 
ment  positions  on  a  "best  qualified"  basis.  There  is 
to  be  an  Acquisition  Managers  List  of  100  officers 
and  a  Senior  Acquisition  Managers  List  of  50  officers 
selected  each  year.  Duty  performance,  leadership 
ability,  and  operational  experience  are  to  be  weighted 
heavily  in  this  selection  process.  Do  you  think  tech¬ 
nical  competence  should  be  considered  as  well? 


Technical  Training/Education  Suggestions 
What  specific  technical  areas  do  you  believe  may 
warrant  increased  emphasis  in  the  training  and  educa¬ 
tion  of  officers  who  will  become  SPO  directors  in  the 
future? 


Appendix  D:  Technical  Advisor  Data  Collection  Sheet 


Program  Manager  Name:  _  Rank: 

I  Technical  Advisor  Name: 


Organization /Prog ram: 


1 .  Program  Manager  Technical  Competence 

1.1  Personal  Expectations  Rating :_ 

Rating  Scale:  1 — Not  very  well 

2 —  Marginally 

3 —  Adequately 

4 —  Fairly  well 

5 —  Extremely  well 

1.2  Model  for  Apprentice  Rating 

Rating  Scale:  1 — Poor  model 

2 —  Fair  model 

3 —  Acceptable  model 

4 —  Pretty  good  model 

5 —  Exceptional  model 

1.3  Talents  Elsewhere  Rating:_ 

Rating  Scale:  1 — Far  more  talented  in  other  areas 

2— -More  talented  in  other  areas 

3 —  Balanced 

4 —  Technical  leaning 

5 —  Talented  primarily  in  technical 
areas 

1.4  Familiarity  Considering  Time  Rating:_ 

Rating  Scale:  1 — Not  familiar  at  all 

2 —  Somewhat  familiar 

3 —  Familiar  with  key  issues 

4— -Quite  familiar  with  key  issues 
5  —  Intimately/ thoroughly  familiar 

with  all  technical  issues 


I 


1.5  Technical  Skills  Rating: _ 

Rating  Scale:  1 — Does  not  have  needed  technical 

skills  at  all 

2 —  Needs  more  technical  skills  to 
perform  well 

3 —  Has  some  technical  skills  but 
more  would  be  helpful 

4 —  Has  most  of  the  needed 
technical  skills 

5 —  Has  all  of  the  technical  skills 
needed  to  perform  extremely  well 

2 .  Program  Manager  Technical  Aptitude 

2.1  Propensity  to  Question  Rating: _ 

Rating  Scale:  1 — Does  not  seek  technical  information 

2 —  Occasionally  inquires  about 
technical  program  aspects 

3 —  Questions  technical  matters 
that  become  issues 

4 —  Tends  to  ask  questions  until 
comfortable  that  someone  under¬ 
stands 

5 —  Strong  tendency  to  ask  questions 
until  technical  issues  are  under¬ 
stood 

2.2  Quickness  in  Learning  Rating: _ 

Rating  Scale:  1 — Does  not  understand  technical 

matters 

2- -Not  a  strong  point,  but 

eventually  understands 

3 —  Average  quickness 

4- -Above  average  quickness 

5 —  Extremely  quick  in  absorbing 
technical  information 


3 .  Importance  of  Technical  Competence 


3.1 


Importance 
Rating  Scale: 


Rating : 

1 -  Not  a  factor  at  all 

2- -Not  a  very  important  factor 

3- -Even  with  other  factors 


4 —  One  of  the  more  important  factors 

5 —  Most  important  consideration 


3.2  Detriment  Potential  Rating: _ 

Rating  Scale:  1 — Would  not  make  a  difference  at  all 

2- -Would  be  unfortunate,  but  not 

seriously  harm  program 

3- -Would  be  a  detriment,  but  not  serious 

4 —  Would  be  likely  to  hinder  the  pro¬ 
gram 

5 —  Would  definitely  seriously  hinder 
program 


Selection  Criterion 

Rating  Scale:  Y — Should  be  included 

N — Should  not  be  included 


Rating : 


5.  Traininq/Education 


S.v 
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